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ABSTRACT 



An evaluation of several of the various methods available for 
analyzing multiprogramming scheduling algorithms is conducted. The 
evaluation consists of an analysis of the capabilities and limitations 
of the methods and an examination of their effectiveness, when used on 
three different types of scheduling algorithms . The three types of 
algorithms are: (l) a single-level round-robin type, (2) a multi-level 

dynamic priority allocation scheme, and (3) a three-level implicit 
priority method proposed by the author for general application, including 
sample-data control use. The results of the evaluation are used in 
formulating a composite investigation procedure that contains the best 
features of the various methods of analysis „ 

The author wishes to express his appreciation to Professor 
Mitchell L. Cotton of the U. S. Naval Postgraduate School for his 
assistance and encouragement during this investigation. 
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1 . 0 Introduction 



The increasing demands upon computing facilities have brought 
about the need for more effective usage of present and proposed facilities . 

As the number of users and user applications grow, it becomes 
apparent that a departure from the normal type of design and usage will 
be necessary, in order to meet expected future demands upon computing 
facilities. This is true in the case of some facilities even now. 

A solution to this problem is the multiprogramming type of system 
design along with closer coupling between man and the computer. Multi- 
programming is defined by Critchlow (6) as, "The time-sharing of a processor 
by many programs operating sequentially. " This means that many object 
programs are in memory (core or auxiliary) , but only one program at a time 
is actually being executed. 

The term multiprocessing is sometimes confused with multiprogramming 
even though the two are generally regarded as distinct concepts. For the 
purpose of this paper multiprocessing will mean, the time-sharing of the 
facilities of a computer by many programs or parts of the same program 
with the aim of achieving maximum use of each and every facility. Multi- 
programming only will be considered in this paper. 

The control of the object programs utilizing a multiprogramming 
system is the job of a supervisory control program, which is generally 
called the executive routine. The main tasks of the executive routine are: 

(l) The determining of which object program to execute next and for how 
long. 
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(2) The exchanging of object programs between primary and secondary 
memory . 

(3) The control of all I/O operations. 

(4) The protection of all portions of memory, belonging to one program, 
from harm by another program . 

The efficient operation of any computer system depends to a large 
extent on the efficiency and correctness of its executive. This is even 
more essential when applied to multiprogramming systems, where each 
and every minute operation of all of the user programs must be scheduled 
and made to occur in a particular sequence that will maximize the systems 
effectiveness . 

A portion of the executive must be allocated the task of scheduling 
these operations according to some algorithm, that will give the optimum 
system performance consistent with the systems goals. When used in 
this paper scheduling will mean, the determination of the sequence in 
which job programs will use the available facilities of the computer 
system. 

This paper will investigate some of the methods of analyzing the 
operation of a scheduling algorithm, point out their capabilities and 
shortcomings , and propose a general scheme for combining the better of 
the methods into an organized analysis of an algorithm. This general 
scheme of analysis will enable a scientific determination of the proper 
scheduling algorithm for a multiprogramming system. 
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The investigation will consist of a general examination of each 
method of analysis and examples of the use of the heuristic and statistical 
methods on several representative types of scheduling algorithms. The 
statistical method will utilize a systems simulator based on the Monte- 
Carlo Method™ 
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2 . 0 Multiprogramming Background 



The terms, time-sharing and multiprogramming, are generally used 
in such a manner as to imply a common definition . This definition is 
taken to be the one given earlier for multiprogramming. For the purposes 
of this paper time-sharing will be used to imply the sharing of a facility 
of the computer system by many programs, on a time basis, and thereby 
will become a tool or subset of multiprogramming. 

2 . 1 Motivations for use of Time-Sharing 

There exist many motivations for the use of time-sharing in a 
computer system. Some of which are: 

(1) Better man-machine interaction. 

(2) More efficient usage of the computer system. 

(3) Faster turn around time (the time between formulation of a program 
and return of the results) „ 

Foremost among these is the desire and need for a much faster 
man-machine interaction rate. Computational speed has been steadily 
increasing, while the methods and speeds of interaction between the 
user and the computer have remained relatively unchanged. The desire 
to increase the man-machine interaction rate stems from the philosophy 
that every man-made system is designed to be a "mechanical extension" 
of man and help him in the performance of his work. Obviously, the 
closer the coupling between the system and man , the more efficiently 
and accurately he is able to utilize the capabilities of the system. 
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The normal type of computer system processes data and solves 
preformulated problems according to predetermined procedures „ During 
the course of computation all alternatives must be considered and covered 
or else the program may not operate properly and erroneous results may be 
obtained . 

This is a very exhaustive and lengthy program formulation process, 
which may consist of many trial runs on the computer before all alterna- 
tives are covered and the program is properly formulated. It would be much 
easier and faster if the user could utilize the capabilities of the computer 
in the formulation of his program. The formulation phase could then take 
the form of a guided trial-and -error process in which the computer would 
turn up flaws in reasoning or reveal unexpected alternative paths in the 
flow of the program . 

By time-’Sharing the computer between several users of the program 
formulation type, not only would the rate of man-machine interaction be 
increased but the utilization of the computer would be increased also, as 
any "dead-time" (unused computer time) , brought about by one user 
analyzing a computer output to him or formulating his next input to the 
computer, could be utilized by another user 

Any one of the above mentioned motivations would seem to be 
Justification for a multiprogramming system and certainly a system that 
could accomplish all three should be very worthwhile indeed. 

2 . 2 System Goals 

Multiprogramming, by its definition, covers a vast area in the 
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computing field. Almost any computer system that time shares any of 
its equipment or units among many programs may lay claim to being a 
multiprogramming system of a sort These various types of multiprogram- ■ 
ming are best segmented into groups according to the goals of each system,, 
There exist several fairly well defined goals and many combinations of 
these with which to segment the types. Some examples of these goals 
are: 

(1) Providing the required service to as many users as possible is one of 
the basic goals , which defines a type of multiprogramming system that is 
receiving more and more attention each day. This is as it should be for 
the computer system is a man made tool and exists for the purpose of 
helping man solve his problems . Any method of providing better service 
to man by one of his tools should receive a maximum of attention and 
effort. 

(2) Providing the maximum utilization of each and every unit or facility 
of the computer system in such a manner as to maximize the efficiency 
of each unit and that of the system as a whole . The efficiency of the 
computer system is a very worthy consideration, especially with the 
vast increase in the usage of computers that is occuring today. Com- 
puter systems now have to service more users , who in turn utilize 
larger and more complex programs than were required a few years back. 

This necessitates the fullest use of all the facilities of the computer 
system . 
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(3) Providing the required service to the control type of users is in 



actuality a subset of the first example, but is of such importance as to 
merit its own separate mention. This type of multiprogramming is seeing 
more and more applications in the military and industrial fields. The 
problems involved are difficult and will require much effort and study. 
For example, how does a computer system provide guaranteed response 
at specific times on a regularly recurring basis to many users? This 
question needs to be answered in order for this type of multiprogramming 
to be fully realized. 

The determination of goals of a multiprogramming system is of 
great concern in the formulation of the executive routine and in fact 
specifies the overall picture of the results that the executive routine 
must obtain . 

2 . 3 Scheduling 

While multiprogramming appears to go a long way toward solving 
some of the computer usage problems , it contains many problem areas 
that will require much thought and effort within its own framework. 

Some of these are outlined by Corbato (4): 

(1) Scheduling algorithms. 

(2) Switching versus swapping of programs. 

(3) Storage allocation . 

(4) Memory and input/output protection. 

(5) Secondary memory usage schemes . 
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Each of these are necessary, integral and interdependent sections 
of a multiprogramming system. The inadequacy of any section can drasti- 
cally lower the effectiveness of the system, regardless of how perfect the 
other sections perform. Of prime importance, however, is the scheduling 
section of the multiprogramming system's executive routine. 

2.3.1 System goals as design considerations 

Because the scheduler decides which user shall receive what service 
and when it, of necessity, must clearly reflect the goal or goals of the 
system. In fact, it may be considered to state the goals in its algorithm. 
For example, if the scheduler considers only the fact that a user or users 
are requesting service and honors them in some manner that does not 
depend on who the user is or where it came from, it then must have as 
its goal the optimization of service to all users without regard to priorities . 
This type of system would not be good for control type user programs, nor 
would it accomplish much toward optimizing the computer system efficiency. 
Consider though, the system that is concerned with computer system effi- 
ciency. Its scheduler would have to consider the past, present, and if 
possible, future work load of each major unit of the computer system. It 
would have to determine the best mix of jobs or job segments that would 
keep each unit as busy as possible. 

It is apparent then, that a very necessary prelude to the design of 
a scheduling section for a multiprogramming system's executive routine, 
is a careful and precise determination of the goals of the system. When 
this is done the task of the scheduler, in most cases, will be outlined 
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and only the problem of accomplishing it remains As in the case of much 
of scientific design, the answer to the question, ’'What exactly is the 
problem," is not only the first, but sometimes the most difficult task 
encountered . 

2.3.2 Overhead Considerations 

The justification of the overhead created by any section of the 
executive routine is a very critical factor. It is even of a more critical 
nature for the scheduler than for other sections, as the scheduler is used 
more often than the other sections of the executive routine. It serves no 
purpose to design a scheduler that considers every aspect of the data 
concerning the user programs and the units of the system, calculates the 
effect on the realization of the goals of the system and goes through a 
complicated process to maximize the realization, only to the end of taking 
up more computer time for the process than was saved (either for the user 
or the system) . This is a clear case of too much overhead and cannot be 
tolerated. The ambitions of the scheduler should be lowered, in this case, 
and a level reached, so that the savings more than exceed the overhead. 

2.3.3 Scheduler Input Considerations 

Another problem area in the design of a scheduler is the inpgf infor- 

# 

mation required for the operation of the scheduler. This must be kept 
w ithin reasonable bounds as some information requirements may be either 
impossible or impractical. For instance it. would be desirable to have 
the expected running time of each user program requesting service, but 
this fact is not known unless the program has been run before with the 
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same input parameters , a rare occurrence indeed . 

2.3.4 Cost -effectiveness Ccnslderatior 5 

In all aspects of the design of a scheduler a cost-effectiveness 
study must be made and the question, "What does this feature buy and 
cost the system and/or user?", must be answered. The cost-effectiveness 
criterion as applied to this answer will determine whether or not the feature 
should be adopted. The cost-effectiveness criterion is another item, that 
is very dependent upon the goals of the system. The system that is 
oriented towards the control type user, for example, will have a cost- 
effectiveness criterion that demands the adoption of any feature enhancing 
the system's ability to guarantee the user service at a specific sampling 
rate . 

2.3.5 Summary of General Scheduler Design Considerations 

A brief summary of the generalized scheduler design problem iss 

(1) There are many various types of schedulers and each is specified by 
the particular goals of the overall system it serves. Therefore, specify 
the goals of the system concisely enough and the type of scheduler and 
its task is specified. 

(2) The overhead of a scheduler must not be allowed to get out of hand 
and the complexity of the scheduler decreased if it does. 

(3) Input requirements to the schedi ler should be carefully examined for 
practicality and necessity. 

(4) A cost-effectiveness study should be made of every feature proposed 
for the scheduler before its acceptance . The cost-effectiveness criterion 

must reflect the goals of the system. 
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3.0 Investigation 



The practical aim in investigating a scheduling system by any form 
of analytical method is usually to analyize the actions of the system and 
try to improve it by appropriate changes . For example , the rate of arrival 
of user requests may be so high that large queues develop, resulting in 
long delays in service for the users, or the rate of arrival requests may 
be so low that the facilities of the computer system are idle for a large 
portion of the time. A change in the scheduling algorithm may be indicated 
in either case. Or it may be that a change in the scheduling algorithm is 
being contemplated and predictions of the amount of change in service 
capabilities are needed. In any event frequent analysis of the operation 
of a scheduler is needed to insure its continuing operation at optimum 
level . 

3 . 1 Methods of Investigation 

There are several methods of investigating the operations of a 
scheduler available to the systems designer, each of which has its own 
particular areas of maximum effectiveness. These methods may be divided 
into the following categories: 

(1) Heuristic approach. 

(2) Queuing theory approach . 

(3) Statistical simulation approach. 

(4) Actual system implementation approach. 

3.1.1 Heuristic Approach 

The heuristic approach generally consists of a mathematical and 

logical examination of the operation of a scheduler without the necessity 
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of gathering large amounts of data for analysis. As a result, it is easier 
to accomplish, but is not as rigorous as other methods, nor does it give as 
complete picture of the operations. It may, however, turn up many problem 
areas and indicate possible directions for further investigation. 

3.1.2 Queuing Theory Approach 

The queuing theory approach is an analytical method that furnishes 
theoretical predictions of the operation of the scheduler. These predictions 
are based on a stochastic model of the scheduler composed of mathematical 
formulas derived from the probabilistic nature of the scheduler environment. 
This model requires inputs of a statistical nature and therefore necessitates 
the investigation of fairly large amounts of data concerning the system. 

For example, the arrival rate pattern of user requests, the service rate 
pattern, the rules governing service and the rules governing the queue 
discipline. The basic weaknesses of the queuing theory approach lie in 
the assumptions that sometimes have to be made concerning the exact 
probabilistic nature of the environment and its inability to handle 
scheduling algorithms that specify much interdependence between the 
various patterns mentioned above. 

3.1.3 Simulation Approach 

The simulation approach is more of a representation of the behavior 
of the scheduler than a mathematical analysis, as is the case with the 
heuristic and queuing theory approaches. The simulation approach does, 
however, utilize the theory of probability and statistics in simulating 
the environment of the scheduler. 
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The simulator is an ideal method to use in place of queuing theory 
when there exists much interdependence between the parameters of the 
scheduler environment. The type of detailed results obtained from simu- 
lation will give a direct qualitative impression of what the behavior of 
the system should look like . 

3.1.4 System Implementation Approach 

The actual implementation approach consists of placing the scheduling 
section in its entirety within the actual multiprogramming executive routine 
and testing its operation within the complete system. This method is not 
to be recommended as a first method of investigation of a scheduler with 
new changes as it is extremely costly in time and effort while also causing 
the disruption of the normal operation of the system during the testing phase. 
3.2 Proposed Composite Investigation Procedure 

Each of the previously mentioned methods has its own place in the 
overall procedure of the investigation of a scheduler and contributes its 
particular type of information to the total collection of intelligence gleaned 
from the investigation. 

An outline of such an overall investigation procedure is as follows: 

(1) Preliminary survey. 

(2) Collection of statistical data. 

(3) Detailed study of the system and effects of any changes to the system. 

(4) Small scale trial of newly proposed system. 

(5) Full scale implementation of proposed system. 
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3.2.1 Preliminary Survey 



A preliminary survey of a scheduling algorithm would entail a brief, 
but careful study of the algorithm and its basic philosophy. This study 
would be of the heuristic type and should reveal the limitations of the 
algorithm and point to avenues of further and more detailed investigation, 
if not to possible modifications themselves. If modifications are indicated, 
they should be roughly assessed and if of merit passed on for a more com- 
prehensive investigation. 

3.2.2 Collection of Statistical Data 

The next step involves the collection of statistical data for use with 
a queuing theory analysis of the scheduler or a simulation model of the 
scheduler. 

The statistical data gathered is generally segmented as follows: 

(1) The arrival pattern data gives the statistical pattern of the arrival of 
user service requests and is generally in the form of a distribution function, 
mean arrival rate and deviation from this rate . 

(2) The service mechanism data concerns itself with the number of users 
that can be serviced during some period of time and the statistical pattern 
of the length of service time, which consists of a distribution function and 
its necessary parameters . 

The arrival pattern data may be separated into requests for various 
types of service and the service data separated in an analogous manner. 
This data is best obtained by proper use of statistical sampling methods . 
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3.2.3 Detailed System Study 



To conduct a detailed study of the scheduler and its operations 
normally requires the use of a simulation model. The queuing theory 
approach will in general be much harder to implement and not give as 
correct data due to the complexity and interdependence of parameters 
found in most scheduling algorithms . 

The correct simulation model will give data that follows closely 
the operations of the actual system. Thereby allowing an investigation 
of the present algorithm to be made under easily varied conditions and 
suggesting modifications that may be checked out in the same manner. 

3.2.4 Small Scale Trial 

The modifications that have passed the investigation of the simula- 
tion model should then be incorporated in the executive routine on a small 
scale and given trial runs to more accurately judge their compatability 
with the rest of the system and their effectivenesses. This procedure 
may be skipped if the modifications are slight and it is readily apparent 
that they will be easily implemented on a full scale basis. However, 
this would be an exception and time and effort will usually be saved by 
a small-scale trial run . 

3.2.5 Full Scale Implementation 

When the modifications have been checked out by a small-scale trial 
run or two, they should be incorporated fully into the executive routine and 
utilized as much as possible for many hours with close observation and 
collection of data in order to judge their effectivenesses. 
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The close adherence to these procedures in the order outlined, will 



allow the adoption of a new scheduler or modification of an old one , with 
full assurance that it will not disrupt the system and will bring about a 
closer realization of the goals of the system. 

3 . 3 Preliminary Survey by Heuristic Analysis 

As stated earlier a preliminary survey of a scheduling algorithm is 
a necessary prelude to further investigation of a new algorithm or changes 
to an old one . This survey may only point the way for further more precise 
investigation or may show the need for changes by itself. In most cases, 
however , it will be advisable to continue the investigation with more 
precise methods. 

For the purpose of this paper a preliminary survey of several different 
types of scheduling algorithms will be presented. These different types of 
algorithms will be representative of algorithms for use in multiprogramming 
systems each having goals of the types described in section 2.2. 

No attempt will be made to prove one algorithm better than another 
as each will have its own distinct goals and will be designed to optimize 
them. It will, instead, be the aim of this section to point out areas of 
investigation and changes that would appear to lead to a better realization 
of the systems goals or a broadening of the goals to include some worth- 
while feature. 

The algorithms presented will generally concern themselves more 
with the user's interests and needs rather than system efficiency as this 
appears to be of the major concern at the present time. 
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Three algorithms will be presented, one of the type developed by 
S.D.C. , one proposed by Corbato (4) , and one conceived by the author. 

3.3.1 Algorithm A 

One of the first categories of scheduling algorithms to examine 
should be one which is intended for use in a multiprogramming system 
with optimization of user service as its goal. An algorithm of the type 
used by System Development Corporation within their "Time-Sharing 
System" at Santa Monica, California is an excellent example of a member 
of this category,, This algorithm will be referred to as Algorithm A here- 
after. 

3. 3. 1.1 Operation of Algorithm A 

The execution of many user programs for a quantum of time (q) in a 
round-robin type of operation, is the basic principle of this scheduling 
algorithm. A round -robin queue is set up for each response cycle with 
the users in the queue being those that requested service during the last 
response cycle time. The quantum or time of operation of each program 
during the cycle is determined by the ratio of the response cycle time to 
the number of users in the queue. The cycle time (t r ) is chosen to be 
approximately the average human response time in order to achieve the 
effect of simultaneous use of the computer by many users. 

The maximum number of users (Nmax) is determined, in part, by the 
minimum allowable quantum time (qmin) . This time is bounded by the 
amount of time required for the average user program to produce a response 
(I/O operation) . 
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If a program does not use up its alloted quantum time in computation, 
but is terminated by a I/O or finished interrupt, the quantum is recalculated 
for the remaining users in the response cycle. The redistributing of the 
remaining time in the response cycle not only increases the efficiency of 
the system, but also allows the larger, slower user programs more execution 
time. If at any time during the cycle no users are in the queue, the system 
goes into an "idle" loop and waits until a user requests service (allowable 
during every interrupt of the quantum clock) . 

Priority of program service during a given interval of time is deter- 
mined by a program's size and its use of low speed I/O equipment. The 
user programs occupy different banks of core memory according to their 
priorities. The priority levels, in the order of service, are: (l) programs 
of less than 16K and not using low speed I/O, (2) programs of less than 
16K and using low speed I/O, or programs between 16K and 32K and not 
using low speed I/O, (3) programs in excess of 32K. 

This scheduling algorithm is well suited and very effective for 
system use of the type that consists, primarily, of tasks that allow a high 
percentage of the user's system time to be taken up by the slow human 
thought and reaction process. Examples of such tasks are: (1) on-line 
debugging, (2) on-line calculation, (3) on-line program building and (4) 
war games . 

The main concern of a scheduling algorithm, when dealing with 
these types of tasks, is to furnish the users with reasonable response to 
service requests and reasonable reaction times. The efficiency of the 
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computer system is considered, but not allowed to become of primary 
importance . 

3. 3. 1.2 Effects of Varying Basic Parameters of Algorithm A 

There are several significant parameters that may be examined 
during a preliminary survey of this algorithm. They are: (1) maximum 
number of users, (2) response times, (3) quantum times and (4) efficiency. 

By the use of variations of one basic formula much information can 
be obtained concerning these parameters. Th° formula is of the "Worst- 
case" type in that it gives a pessimistic view of the operations based 
on values of the formula parameters that represent the worst conditions 
possible. This type of analysis is very helpful, as it shows the lower 
bound of service capabilities and insures good operation if expectations 
do not exceed this lower bound by significant amounts . 

For determining the maximum number of users allowable with the 
minimum quantum, with the percentage of overhead (n) and the "Worst- 
case" value of the time it takes to transfer a program from secondary to 
primary memory or vice versa (t g ) either known or approximated, the 
following formulas apply. 

Equation 1 Equation 2 

Nmax = Ml-n) Nmax = fc r (l-n) 

2t + qmin 2t_ 

S a 

These formulas are derived by analyzing the basic cycle of operations 
concerning one user during a response cycle. 
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First it is determined, which user is to be executed for the particular 



quantum of time and his program is brought in from secondary memory, 
after the last operating program is transferred to secondary memory. This 
takes an amount of time equal to 2t g . The program is then operated for 
one quantum (q) . Of course , deciding which program to bring in and other 
executive functions take up a certain percentage of the cycle time (t T ) and 
have to be accounted for under the heading of overhead (n) . 

Clearly, then, the denominator of the Equation i represents the time 
necessary to execute one user during the response cycle while the numerator 
represents the effective operation time available during the response cycle . 
It must be emphasized thatNmax is the maximum number of active users in 
the system i.e. , the number in the queue , There may be many more 
actually using the system as long as not more than Nmax request service 
during the same response cycle . 

In order to investigate ways of increasing Nmax, the effects of 
changes in the various parameters of Equation 1 on Nmax must be analyzed. 
It will make the investigative task simpler, if the parameters are broken 
up’ into groups according to whether they are easily changed by a software 
approach or require hardware modifications to induce significant changes. 

The swap time (t g ) is the only parameter that would most likely 
require hardware modifications to measurably change it. For instance, 
the swap time may be decreased and Nmax increased by increasing the 
transfer rate between primary and secondary memory, allowing two or more 
programs in memory, or both. Increasing the transfer rate is obviously a 
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hardware change and is usually a major one if substantial increase in 
rate is to be accomplished. Allowing two programs in memory, at first 
glance, does not seem to require hardware modifications, but on further 
analysis it will be seen to require memory protection to prevent inter- 
action between the two programs and dynamic memory relocatability , in 
order to achieve the desired results. 

The main thought concerning hardware changes , whether they are 
modifications to existing hardware or design features for proposed equip- 
ment, is to be sure that the increase in capability is substantial enough 
to warrant the cost of the change. 

The technique of showing graphically the capability increase versus 
change in parameter values is an invaluable tool to use in determining 
whether or not further study is warranted . This technique is illustrated 
in Figure 1 where curve A represents Nmax versus primary to secondary 
memory transmission rate and curves B and C show the relationships 
between qmin and t r , respectively, and Nmax. The latter two variables, 
qmin and t f , are variable by software means and depend more on user 
wants and needs for their values than on the computer system limitations. 
This means that the cost of their change will be in terms of changes in 
some user service parameters rather than monetary values or man-hours. 

The estimation of the effect of allowing two or more programs in 
memory at one time actually involves the modification of Equation 1 . If 
memory sharing is allowed and qmin is substantially less than 2t s , then 
the limiting factor in the denominator of Equation 1 becomes 2t g and 
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Figure 1 



Effects of Variations in 
Algorithm A Parameters 



Nmax = t „(l~ n )/ (2t +q) 
r s 
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Equation 1 shifts to the form of Equation 2 . The minimum quantum {qmin) 
effectively vanishes because concurrency is now possible between the 
transferring of one program and the execution of another. If qmin were 
much larger than 2t g , then 2t s would likewise effectively vanish, but this 
will hardly ever be the case. 

Whether or not 2t^ is substantially larger than qmin depends upon 
the distribution of user requests and can only be roughly estimated without 
such data. From observations of the "Time-Sharing System" at S.D.C. , 
2t s does not appear to be much larger than qmin and therefore the effect 
of memory sharing on Nmax does not seem to warrant its cost. Of course, 
this applies to this particular system and its users and the outcome of 
the analysis might be entirely different for another system and its users. 
The important thing is that the method of analysis will remain the same 

For a complete analysis of the algorithm, other pertinent charac- 
teristics, such as, response time and computational efficiency, must be 
investigated , 

Response time , for this algorithm, may be considered t r , for worst 
case analysis. In other words, a user requesting service immediately 
after a new response cycle had started would have to wait a maximum of 
time (t r ) until he was accepted for service, while one which requested 
service at the end of a cycle would have to wait a minimum of time. 
Another form of Equation 1 may be used (Equation 3) for this analysis 
and graphical results illustrated by curve C of Figure 1 . 
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Equation 3 



t f = N(2t s +q)/(l-n) 

As may be seen by an examination of the equations and the curves 

of Figure 1 , N and t are linearly dependent (if 2t and q remain constant) 

r 

and therefore each is very sensitive to any change in the other. As each 
increases without bound, with any increase in the other, some upper limit 
must be set for one of them in order to contain the investigation in a 
workable area. This limit is more easily placed on t r . The response time 
should not be allowed to exceed a value that would appreciably degredate 
the service needed by the type of user for whom the system has been 
designed. In the case of this particular algorithm, the maximum t is an 
anproximation of the human response interval. This then confines the 
problem to a smaller working area and families of curves A, B, and C may 
be drawn within this area for closer analysis. 

It is clearly shown by the graph that if the major concern is to 
maximize the number of users, then t must be set at its maximum value. 

On the other hand, if servicing a user with particular needs as to response 
time is of primary interest then N must be restricted to a value that will 
allow the needed t f . In most cases a combination of these goals is present 
and a compromise is reached between Nmax and t f max with the other para- 
meters being varied in order to ease the conflicts. This is the case with 
the particular algorithm in question as t is restricted to a maximum value, 
but allowed to vary up to this maximum as the number of users dictate. 

The quantum time is allowed to vary, but has a minimum value that 

is normally arrived at by a compromise between computational efficiency 
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and maximizing the number of users. Clearly as q is decreased the 
number of users allowed (N) goes up, but the more often a program is 
transferred in and out of memory the more overhead is increased, swap 

time per program is increased and computational efficiency is decreased. 

■ 1 c . ' 

The term computational efficiency is taken to be a measure of the ratio of 

- c 

1 - . 

the time a user program spends in the computational phase to the time it 
spends in preparing for the computational phase. The latter time does not 

T v i ' n 

include time spent waiting for a normal user turn The following equation 

I 1 J • 

is one method of expressing computational efficiency (C.E.). 

1 

Equation 4 

C.E. = q(l-nq) 

2t s 

The parameter n represents the portion of overhead that may be 
considered taken up by each user everytim® his program runs for a quantum 

_ ■ ^ i 

of time. Because of the fact that t s is not easily variable computational 

•C < . , . - I.6K .p " 

efficiency depends to a great extent on quantum time and is maximum when 

• u o. ■ ■ ■ ' 1 ^ ' 

q is maximum. 

,, , r _- ,\.H : V; „f . X - 

It is important to bear in mind that rarely will a multiprogramming 
system achieve a computational efficiency greater than 50% and most do 
not even reach this. Computational efficiency, therefore, should be 
maximized, but not at the expense of user service and should not be used 
as a major comparison parameter between systems. 

The minimum quantum time is determined not only by consideration 
of computational efficiency (as illustrated in Equation 4) and the number 
of users allowed, but also by a consideration of obtaining the maximum 
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use out of every assigned quantum of time. It does not help to assign 
a user a larger quantum of time in order to gain better efficiency only 
to have him terminate his quantum by an I/O operation after only a few 
milliseconds. Nor does it help to assign a user a smaller quantum and 
have him need only a little more to come to a logical termination. These 
situations can not be avoided for every user, but by a careful study of 
user distributions as far as program run time between I/O operations is 
concerned and anslysis of the effects of varying q, a suitable compromise 
between efficiency, and user service may be reached. 

3. 3. 1.3 Summary of Analysis of Algorithm A 

The following comprise a brief summary of the heuristic analysis 
of Algorithm A. 

(1) Either an excellent intuitive feel for the expected distribution of 

such thing as, user arrival rates, time between I/O operations and response 
time requirements or else actual distribution data should be available for 
a preliminary analysis. 

(2) The user service oriented scheduler must set a maximum limit on t r 
that is based primarily on user response requirements . 

(3) A minimum limit on quantum time must be set consistent with proper 
utilization of the quantum (based on user time between l/0"s distribution), 
number of users allowed in each response cycle and computational effi- 
ciency. The emphasis should be placed on the first two conditions. 

(4) Unless much faster transmission rates between primary and secondary 
memory are available, memory sharing does not appear to increase the 
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capabilities of the system enough to warrant its implementation. 

(5) A simulation study of the effects of not allowing early I/O terminations 
to cause a recalculation of q would be helpful in determining if the 
overhead involved were prohibitive . 

(6) A simulation study of the effects on computational efficiency and user 
service due to the modification of the algorithm to allow background jobs 
to fill in when the system has no other users would also be advisable. 
3.3.2 Algorithm B 

The next type of algorithm presented for analysis is one which also 
has as its goal optimum user service, but does give consideration to 
computational efficiency. 

This algorithm (3) was developed at Massachusetts Institute of 
Technology and was one of the forerunners of the field. For the purpose 
of this paper the algorithm will be designated Algorithm B. 

Algorithm B is what is considered a multilevel scheduling algorithm, 
as the scheduling process sets up multiple level queues of programs 
waiting to be serviced. 

3. 3. 2.1 Operation of Algorithm B 

The multilevel scheduling algorithm is implemented in the following 
manner: 

(1) Each user program is assigned to the L 0 th level priority queue, as it 
enters the system. 

Equation 5 

L q = /Log 2 ( / w p /Wq_/ + 1)_7 
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number of words in user program 



w = number of words that can be transferred in and out of primary 

q 

memory during one quantum (q) of tir e 
Integral part of A 

(2) The program at the head of the lowest level queue (L) is executed for up 
to 2 quanta of time and if it is not completed is placed at the end of the 
L+ bilevel queue to be executed for up to 2 _ quanta of time later on 

After execution of all of the L t , level programs for 2^ quanta , programs in 
the L+l^ level are executed. If a program at the L+ X th level is being 
executed and the level becomes occupied by an active program, the 
presently executing program is placed at the head of the L+l^ level queue 
and the L t h level program is brought in and executed . If there are no 
levels active, background jobs are brought in and executed until a level 
becomes active, thereby increasing computational efficiency and lowering 
idle time . 

3. 3. 2. 2 Effects of Varying Basic Parameters of Algorithm B 

From Equation 5 it can be seen that program size is an essential 
input into Algorithm B and therefore requires that user programs be pre- 
compiled . 

Further study of the equation shows that L c can never be less than 
zero due to the fact that (w p /wq+l) has one as its lower limit. With 
(wp/wq) q representing the swap time (2t g ) of the program, then L 0 is 
always at least slightly greater than Log2 {2t s /q)„ This coupled with 
the practice of executing the program for 2^°{q) or, as 
(2 L °92^ 2t s/q)) (q) = 2t s , for greater than the swap time of the program. 
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insures a computational efficiency of at least 50% . 



A computational efficiency of at least 50% arises from this analysis 
only if overhead time is neglected. As overhead can never be neglected , 
an overhead factor {n) must be included in any equation concerning 
efficiency. 

Equation 6 may be taken to represent computational efficiency with 
overhead included for programs operating from the level queue. 

Equation 6 

C.E. = Nq 2L (l-n) = ( Zw p /w q _7 + D (1-n) 

N CZ w p/wq_y ) q ( Z w p/ w q_/ ) 

Where N = number of users at Level L 

n = fraction of overhead involved for each userZquantum 

As may be seen Equation 6 reduces to the point where computational 
efficiency appears to be independent of q and N . This is not strictly true 
as n is a proportional parameter and is based on a particular q. This then 
imposes a limit on q that requires it to remain large enough to keep the 
overhead to a low proportion of q and computational efficiency close to 
50%. 

Another equation that is helpful in analyzing Algorithm B is one 
that describes the calculation of the worst-case response tim e (tj.) for 
each user when all N users are on one level. 

Equation 7 

t r = 2Nq CZ w pZw q _Z+ 1) “ Nqm 
= Nq 2 ( Zw p Zw q _Z + 1) - n 
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Equation 7 is arrived at by considering that q ( Z w p/ w q_/ + 1) is 
both the swap time and the execution time for each program and that qn 
is the overhead time for each program. With N programs at the particular 
level, N times the sum of the swap time, execution time and the overhead 
time will determine the length of time any user has to wait between execu- 
tions of his program. 

Equation 7 shows that response time is directly proportional to both 
the number of users and quantum time. Quantum time has already been 
shown to have a lower bound fixed by efficiency considerations and now 
response time (or Nmax) considerations create an upper bound. It is very 
apparent that a compromise must be reached between the number of users 
allowed at each level (Nmax) and the required response time (t r ) , just as 
in the case of Algorithm A. 

A graphical picture of the various relationships expressed in Equation 
7 is shown in Figure 2 . 

3. 3. 2. 3 Summary of Analysis of Algorithm B 

The following statements may be used to summarize the preliminary 
survey of Algorithm B . 

(1) The quantum time is chosen in approximately the same fashion as for 
Algorithm A with the lower bound set by efficiency requirements and the 
upper bound set by maximum allowable user considerations. 

(2) Response time and maximum number of users must be determined by a 
compromise between the optimization of each . 
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(3) All of the equations concern one level only and consider all programs 
in the system to be on this level for worst-case analysis. To accomplish 
a proper analysis considering the users spread about on multilevels is a 
complex undertaking and is best left to a simulation study . 

(4) It is possible for users with large programs to get pushed up to a high 
level and never get to run at all if there exist many small users with 
relatively short times between I/O operations . This could be corrected 
by insuring that every user in the system was allowed to run at least once 
during some cyclic time regardless of his level position „ 

(5) The large user program, which has fast reaction time (time between 
l/0's) is unduly restricted by its size as it can never go to a lower level 
than the one it entered upon and may use only a small portion of its 
alloted execution time , while still having to wait the full tj. specified 

by its high level before its next execution turn. There should exist some 
compromise between raising the level for larger and longer running programs 
and lowering it for programs that consistantly terminate their execution 
time early. 

(6) The background user approach represents a goodly increase in efficiency 
particularly if the type of foreground users are those who require periods 

of thought process between I/O operations and thereby leave a fair amount 
of idle time for use by the background user. 

3.3.3 Algorithm C 

There exists a need for a scheduling algorithm that can handle 
efficiently a variety of user types such as: (1) production job user, 
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(2) man-machine communications user and (3) sample-data control 
user. 

A production job type user is typified by his ability to encounter 
fairly long delays in obtaining service without harmful effects. 

The man-machine communications user needs to have a response 
time somewhere close to the human reaction time, but is not hurt badly 
if he misses a turn or two occasionally. 

Sample data control users are very critical concerning response 
time. These users must be assured of a certain fixed for each different 
user) interval between its samples or execution turns. This puts an 
extreme strain on the scheduler and requires a very stringent priority 
queuing scheme. 

The task of trying to serve all three of these types of users 
necessitates the design of a scheduler that tries to meet all three of 
the goals outlined in Section 2.2. Of course., any attempt to maximize 
three different forms of service as one is bound to result in a system that 
compromises and does not fully maximize each individual type of service. 
This is to be expected and resultant service degreaation to any one type 
of user must be accepted as the price required for a generalized rather than 
specialized system. 

The algorithm proposed for this generalized type of system. 

Algorithm C , is a three-level scheduling algorithm, where the various 
levels are defined by the type of users assigned to each one. The levels 
and their associated programs have inherent priorities and reside in 
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different types of storage when awaiting service. Figure 3 illustrates 
the level organization. 

3. 3. 3.1 Operation of Algorithm C 

The operation of the scheduling algorithm is as follows; 

(1) At the beginning of a response cycle, the number of user requests 
for each level is determined (only one user each from levels I and III are 
allowed per cycle) . 

(2) The sample rate and execution time of the level I user will have to be 
known and the quantum time (qj) will be equal to or slightly greater than 
his execution time, just as long as the reciprocal of the sample rate (t s ) 
is an integral multiple of the quantum time. This is in order to be a' e 
to insure the level I user of service at exactly his specified sample rate, 

(3) The total execution time of the level I user is subtracted from the 
cycle time (which is also an integral multiple of calculated q^) and the 
remaining time is allocated to level II and III users on the basis of a q2 
(integral multiple of q^) basis. The quantity, q-, , is not allowed to 
exceed t s . 

(4) Only one is given to the level III user unless there are not enough 
level II users to fully utilize the allotted time, in which case, the level 
III user is given the remaining time. 

(5) Early terminations of q *2 by level II users I/O operations are not ' 
allowed to change the structure of the queue nor cause recalculation of 
q^ as this would result in the inability to insure the level I user of a 
specified sampling rate . 
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(6) If a level I user is requesting service, a new cycle execution may 
not begin except at the time specified, by the level I users sample rate, 
for the next level I user sample. 

Figure 4 shows the manner in which the queue is formed for 
Algorithm C . 

(7) The queue should both begin and end with a level I user in order that 
overhead time involved in setting up the queue, etc , will occur between 
level I samples and therefore not unduly degredate sampling rates. 

As the number of users from levels I and III are restricted to one 
each per response cycle, it is fairly easy to express, in equation form, 
a relationship concerning the number of level II users. This equation 
is specified by the following parameters: 

(1) t c - time of one response cycle. 

(2) n - percentage of overhead per cycle . 

(3) N 2 - number of level II users . 

(4) - pseudo number of level I users or sample rate divided 

by cycle time (l/t s t c ). 

(5) t 2 - time to transfer an average level II user to or from core 

memory. 

(6) tg - time to transfer an average level III user to or from core 

memory. 

(7) q 2 - quantum time for level II users (Mq^, where M is an 

integer) . 
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The number of level II users allowed (N^,) is equal to the amount: 
of effective cycle time (t c (l-n)), minus the time to transfer user programs 
during the cycle {iN^Stn^t^, N 2=1 ) , divided by the level II quantum time 
(Mqj), minus the number of level III users (l) and the pseudo number of 
level I useis prorated to a number utilizing a level II quantum time (N ,/M) 
From this description of the number of level II users comes: 

Equation 8 



N 2 = t c (l-n)-N 2 2t 2 -2t 3 

Mq x 



= (1+N j/M) 



or 



Equation 9 



N 2 = t c (l-n)“2t3-qj (M+N^) 
Mqj+2t 2 



3. 3. 3.2 Effects of Varying Basic Parameters of Algorithm C 

As might be expected from the description of the operation of the 
three level scheduling algorithm, the specifications concerning the level l 
user’s needs dominate the algorithm. This is reflected in Equation 9 also. 
Note that the factor q M is equal to the time between level I user samples 
(t_), the term N is equal to l/t s t c and q, is in reality the execution time 
of the level I user. This leaves only the transfer times (t 2 and tg) and 
the cycle time that may be varied to maximize N 2 . The transfer times 
are essentially hardware limited variables and therefore are costly to 
change. Cycle time may be increased to allow' more level II users, but 
will cause a corresponding increase in level II users response time as t 
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is the response time for all but level I users , In short the critical 
factors in this algorithm are controlled by the level I user and variations 
allowed are few. This will, in general, be the case for any algorithm that 
concerns itself with control tvpe users, as there is very little flexibility 
in their service needs . 

There are several limiting factors inherent in this algorithm and 
the services that it can provide . First of all consider the limitation on 
the maximum sample rate allowed the level I user, One of the factors 
connected with the sample rate limitation is the average time necessary 
to transfer a level II user to core and let him accomplish some worth 
while computation „ 

With a fairly fast computer having two high speed I/O channels 
and capable of having several programs in memory at one time, this 
time may be controlled to where it is not the limiting factor. With the 
above mentioned capabilities the transfer times are , on the average , 
effectively "swallowed" by the quantum of computational time due to the 
allowance of concurrency of compute, input and output operations. 

The main limiting factor then comes to be the overhead time between 
response cycles as, with the level I user having a quantum of execution 
time at the beginning and the end of the cycle, the overhead time effec- 
tively represents the sample time. It is estimated that this value could 
be kept in the neighborhood of five milliseconds for computers with 
average instruction times of around three microseconds. This would allow 
200 samples per second or effective sampling of 100 cycle per second 
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functions. A goodly number of control problems may be specified within 
this frequency range and investigated under multiprogramming. 

3 . 3 . 3 . 3 Limitations of Algorithm C 

The major drawback of the algorithm is that, as shown above, 
two high speed I/O channels, extensive and flexible memory protection 
and large core memory is required in order to achieve decent service 
capabilities for the level I users. These requirements are extremely 
expensive and are beyond the reach of many installations . One further 
drawback is the reduction of the level II and III users quantum time in 
order to gain better sample rates for the level I user. 

3 . 3 . 3 . 4 P roposed Changes to Algorithm C 

A possible solution to the problem would involve forming the cycle 
for the level II and level III users alone and then allowing the level I user 
to interrupt, when ever and for as long as he needs, via a separate quantum 
clock. As the level I user would normally not need a very long quantum, 
this would not increase the cycle time beyond reason and would allow the 
level II and III users to stay in core long enough to achieve meaningful 
computation. In addition the extra high speed I/O channel would not be 
needed, as with a larger q^ the transfer times could be accomplished, 
for the most part, concurrently with computation and would not degredate 
level I user service in any case. A further advantage is that computa- 
tional efficiency would go up due to the overhead involved in transfers, 
etc. becoming a smaller percentage of . 
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3,3,3„5 Summary of Analysis of Algorithm C 



It should be apparent now that the problems involved in including 
capability to service one-line control type users in a multiprogramming 
system are many and no easy solution exists that will provide good 
service for all types of users. Probably the easiest solution to implement 
is to provide two separate algorithms (one for all three levels of users 
and one for level II and III users only) for the system that may be switched 
in or out depending whether or not control type users are scheduled for the 
particular time concerned or not. This though requires strict scheduling 
of blocks of time for different types of users and should be considered 
an interim measure only. 

The algorithms involved in the on-line control class are sufficiently 
complex to be very hard to analyze by simple methods and therefore lend 
themselves readily to analysis by simulation methods. The main concern 
in the simulation analysis should be determining what values for the para- 
meters of the algorithm in question give the best level I user service 
(with respect to sample rate) and, at the same time, the least degredation 
to level II and III user service. The investigation should be concerned 
with effects on computational efficiency also. 

There exists a great need for work in this area of multiprogramming 
as universities, research laboratories and even industry would benefit 
greatly by being able to solve all of their computing needs (control needs 
included) with one computer system and yet allow each user to feel that 
he has the system at his disposal at any time. 
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3.4 Detailed System Study by Simulation 



As stated earlier , the simulation approach to the analysis of a 
particular scheduling algorithm is characterized by its detailed picture 
of the operations of the scheduler and its ability to provide a method 
that allows the analysis of an algorithm with a high degree interdependence 
between its parameters „ 

The algorithms mentioned in section 3.3 all contain a certain degree 
of interdependence among their parameters and, while a heuristic approach 
may bring forth much valuable information, a detailed study of the opera- 
tions of the algorithms should be conducted by simulation techniques before 
decisions concerning changes or acceptances are made. 

3-4.1 F eatu re s of The Simulator 

The simulator used for the purpose of this paper is very flexible and 
can be used to examine any part of the system's operation that may be 
desired . 

3 4.1.1 Input Data features 

The input data consists of such things as; 

(1) User distributions as pertain to job arrival times and various service 
rates . 

(2) Job types. 

(3) Number of user cnannels . 

(4) Type of scheduler to be used. 

(5) Parameter of algorithm to be varied. 

(5j Type of output desired. 
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(7) Length of run (time) „ 

An example of the input data to the simulator may be seen at the end of 
the program contained in Appendix II . 

The part of the input data concerning the user distributions and job 
types are converted by the simulator to job specifications for each user 
station. The specifications contain the following information: 

(1) lob numoe r-chronological number of the job generated. 

(2) Clock time the time of generation of the job. 

(3) Station numb er- the number of the user station for which the job was 
generated . 

(4) Jo b type the type of job generated (specifies the particular set of 
means used to generate the job). 

(5) Arrival time -the time that the job will request service. 

(6) Load time -the amount of time needed to load the job. 

(7) Active time - the amount of time the job should spend actually computing. 

(8) I/O time - the amount of time the job will spend in an I/O phase. 

(9) Re;.eats °the number of times the active and I/O phases will be 
repeated . 

(10) Size° the number of memory cells required to contain the job. 

An example of the generated job data is contained in Figure 6. 

3 . 4 . i „ 2 Output Data Features 

The output data may be arranged in convenient tabular form or in 
graphical form depending on which is desired. This capability allows the 
investigator wide latitude in choosing the data format that will be easier 
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to analyze for each output parameter . 

It will be possible to form comparisons between various schedulers 
using the same user distributions and determine which one best suits the 
needs of the system, and its expected users by use of the simulator The 
simulator also allows the study of the effect of varying any algorithm 
parameter and can present tabulated or graphical output data for ease of 
comparison between various values of the parameter in question 

For analysis of scheduler operations,, the simulator output data 
contains. 

(1) Variable parameter -Values of the parameter that, is varied during the 
run . 

( 2 ) Cycle count . The number of response cycles that occurred. 

(3) Average c ycle time The average amount of time taken by a response 
cycle . 

(4) Average number in queue -The average number of active users during a 
response cycle . 

(5) Average quantum -The average amount of time taken by one quantum 

(6) Average overhead -The average amount of time taken up by scheduler 
overhead during one cycle 

! 7 ) Computat i onal efficiency -The average .amount of time spent actually 
computing per response cycle, divided by the average cycle time 
(5) Average exchange overbead -The average amount of exchanger overhead 
per response cycle. 
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(9) Average exchange time -The average amount of time taken up by loading 



and exchanging per response cycle (does not include overhead), 

(10) Average overload -The average number of jobs that were unable to 
obtain service due to the queue being full, per cycle. 

(11) Maximum quantunv The maximum amount of time taken by one quantum, 

(12) Maximum cycle time -The maximum amount of time taken by one res 
ponse cycle . 

(13) Maximum number in q^eue- The maximum number of active users during 
one response cycle. 

(.4) Maximum. overhead -The maximum amount of scheduler overhead during 
one response cycle 

(15) Maximum overload -The maximum number of jobs unable to obtain 
service during one response cycle. 

(16) Maximum, number of stations -The maximum number of user stations 
allowed . 

(17) R equests serviced -The number of user requests for service honored 
Figure 7 contains an example of the output format of the simulator. 

As may be seen the simulator output easily furnishes enough informa 
tion for the proper analysis of the operation of a scheduling algorithm 

For further and more detailed information concerning the operation 
of the simulator consult Appendix I. 

3.4.2 Example of Simulator Operation 

In order to demonstrate the capabilities and operation ol the simulator, 
an analysis of the scheduling algorithm (Algorithm A) used by System 
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Development Corporation in their "Time-Sharing System" at Santa Monica, 



California will be conducted using the simulator. A flow diagram of the 
scheduler section is contained in Figure 5. 

3 . 4 . 2 . 1 Input Data 

The input data is based on observations of the operations of the 
"Time-Sharing System" and while the data is not the result of a thorough 
statistical study of the system, it will be sufficient for the purpose of 
this example , 

3 . 4 . 2 . 2 Variations of the Algorithm To Be Considered 

The preliminary analysis of Algorithm A in section 3.3.1 outlined 
several variations of the algorithm that would be worth while investigating 
by simulation techniques. These and others will be analyzed by the use 
of the simulator and it will be determined if the variations would further 
the achievement of the system's goals or not. 

The variations to be investigated are: 

(1) Not allowing an early termination of a program's allotted quantum to 
cause a recalculation and redistribution of the remaining response cycle 
time. Early termination means the termination of a job's active time by 
I/O operations or quitting before the normal quantum is complete 

(2) Permitting background user type jobs to be run when nothing else is 
active . 

(3) Varying the maximum number of user stations allowed in order to 
determine the best, level of operation . 
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FIGURE 5 

Flovr diagram of the scheduler seotion 
of the simulator 
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(4) Varying the rate of arrival of job requests by varying the arrival time 



average in order to check on the overload conditions 

The variations will be brought, about by both changes to the algorithm 
and changes to the input data. 

3.4.23 E ffects of Early Quantum Termination 

In order to analyze this problem, the simulator was used to simulate 
a one hour normal run of the "Time-Sharing System" with its normal type 
of user activity. A one hour run each was made both with early termina- 
tions allowed to cause redistribution of remaining cycle time and without 
this capability. The input data was the same for each run and therefore 
each run was made within the exact same user request environment An 
example of the job data is contained in Figure 6. 

As the goal for Algorithm A is to give the best, possible service to 
as many users as possible, the output data must be analyzed with this 
in mind. The run that accomplishes this goal best: will determine the 
variation to be recommended for acceptance . 

The first line of each section of Figure 7 is the output data from 
simulator run one where early termination has control over redistributing 
cycle time, while the second line of each section is from run two, where 
early termination does not have control 

An examination of the data contained in Figure 7 shows that run one 
honored more service requests, gave a larger average quantum and had a 
better computational efficiency than run two. The only advantage run two 
had over run one was the decrease in average scheduler overhead, which 
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GENERATED JOB DATA 
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FIGURE 6 (CONTINUED) 
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FIGURE 6 (CONTINUED) 
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SIMULATOR OUTPUT DATA 
WITH EARLY TERMINATIONS ALLOWED, 

NOT ALLOWED AND BACKGROUND JOBS PERMITTED 



UJ 

UJO 

oz — m ro 

<<'JJ ro r— in 

ccjzz- >o o o 

tuu — o •— . cr 



>XK 


• 


• 


• 


<yj 




. — 





UJUJO 
















OCX 
















CIXLLI 


in 


CM 


o 


OOCv ' 


O 


O 


O 


c£ < X 


>0 


U* 


m 


2TUJ 


O 


0 


.O 


ojxcs: 


o 


O 


O 


OO 


O 


0 


O 


>OLU 


o 


O 


O 




O 


0 


O 


<x> 


• 


• 


• 


h-> 


• 


• 


• 


LUO 








<c£ 


0 s 


ON 


NO 


-J 








H* UJ 


O 


in 


O 


< 








LOCO 








2T , 
















CJ> 
















>-HO 


00 


o 


CO 


OO 


O 


0 


O 




O' 


o 


I s - 


2 : z 


O 


0 


O 


<LU 


r— 


r— 


f— 


oq:q 


O 


0 


O 


1— 


ro 


ro 


o 


51 LU >— < 


O 


0 


O 




• 


• 


• 


^CCr- 


• 


• 


• 


CL • 


o 


r- 


NO 


X5T< 


O 


0 


O 


SILL 


CM 


*— 1 


CM 


<X>I- 


CM 


CM 


CM 


OIL 








SIZOO 








OLU 
















UJO 


O 


o 


O 


Q 


O 


O 


O 


o< 


NO 


CM 


r- 


X< 


O 


O 


O 


<LU 


lD 


r— 


LO 


OO 


O 


O 


O 


ozx 


o 


o 


o 


>L_J 


O 


O 


O 


UJOC 


• 


• 


• 


►—4 (V 


• 


• 


• 


>UJ 








X UJ 


r— > 


•— 




<> 








<> 








o 






• 


5:0 








UJ5I 


o 


c> 


d 


0 


O 


O 


O 


OO 


o 


CD 


o 


SI< 


CM 


O 


CM 


<i— 


-d 


CM 


d" 


XUJ 


I s - 


r— 


ro 




o 


o 


o 


SHE 


CN 


O 


O' 


UJ< 


• 


• 


• 


H-.QC 


• 


• 


• 


>o 








X LLI 








<o 








<> 
















510 








LU UJ 


CNJ 


CM 


o 


UJ 


O 


O 


0 




r— 


O 


d 


X) 


O 


O 


0 


<LULU 


PO 


CM 


o 


‘ XqZlu 


O 


O 


0 


OC CQ O 


CNJ 


O 


LO 


SZlDO 


O 


O 


0 


ujsia 


• 


• 




4 CC 0 


• 


• 


• 


>D 


,*0 


JT 


ro 


X2l 


O 


O 


0 








*— 


CDZ 


CM 


CNJ 


<M 


>— * 








5TZ>- 








UJ 


CM 


rO 


N- 


SI 


O 


ON 


to 


OLU 


.-d- 


O 


ro 


O 


CD 


CM 


O 


<*JLU 


»— 


-d 


O 


SILLf 


O 


d 


I s - 


q:uz: 


rO 


ro 


ro 


•— -JOJ 


ro 


d 


d 


LU> »— « 


• 


• 


• 


xus: 


• 


• 


• 


>Uh 


CNJ 


CM 


CM 


o--* 


00 


CD 


00 


< 








SZOH- 









ILJh- 


0 


O 


O 


sis: 


O 


O 


O 


J 2 


0 


O 


O 


DD 


O 


O 


O 


OX) 


0 


O 


O 


Zu h- 


O 


O 


O 


>0 


0 


O 


O 




CM 


CM 


CM 


00 


• 


• 


• 


x< 


• 


• 


• 




in 


d 


O 


<0 










0 


ro 


in 


SIC? 










I s - 


in 


1 s - 












r ~ 


•— 


•— 










UJOZ 


0 


0 


0 


a 


NO 


I s - 


O 


-JUJ 


0 


0 


0 


UJ< 


O 


O 


O 


CQ»- 


0 


0 


0 


00 


O 


O 


O 


<LU 


0 


0 


0 


<0 


O 


O 


O 


— tS 


• 


• 


• 


oca: 


• 


• 


• 


oC< 


r— 


CM 


ro 


UJLU 








<;a: 








>> 








>< 








<0 









Q. 



53 



was offset by the fact that: the scheduler overhead represents hut a small 
part of the total overhead (scheduler overhead, exchanger overhead and 
exchange time) and therefore the total decrease was negligible 

The outcome of the simulator analysis clearly show's that the varia- 
tion of the algorithm that allows early termination to cause a redistribu- 
tion of remaining response cycle time appears to give better service to 
the users and therefore should be implemented within Algorithm A 
3 4.2,4 Effects of Allowing Background Users 

A simulated one hour run (run one) was made by the simulator 
without background users and another (run three) was made with back 
ground users in order to determine if background users should be allowed 
or not. Lines one and three of each section of Figure 7 represent the 
simulator output for runs one and three respectively. 

The data shows that, while the number of service requests honored 
while allowing background users was not as high as while not allowing 
them, the reduction was small and the computational efficiency went up. 
That the computational efficiency went up only slightly, is in part due 
to the system already being used very efficiently (for a multiprogramming 
system) and therefore not leaving much inactive time for the background 
user to take advantage of , 

It is r ecommended that background users be allowed , even though 
the efficiency gain is slight, in order that large, long computing jobs 
(such as compilations) may be run as background and not degredate the 
other users' service as much as they would if they were run as regular 
time-sharing jobs 
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3 . 4 . 2 . 5 Effects of Varying Maximum Numbe r of Users 



To study the effect of variations in the number of users allowed , 
ten one hour runs were simulated. Tne number of users tor each run was 
incremented by live at the completion of each run. The range of users 
allowed was from 5 to 50. 

As the variable parameter was the number of users , the number for 
each run may be found in the variable parameter column of the first section 
of Figure 8 as well as in the maximum number of stations column of the 
second section of Figure 8 . 

As w'as expected, the results of the simulation study showed that, 
as the number of user stations allowed went up, the total overhead went 
up, the computational efficiency w-ent down and the response lime 
(average cycle time) increased. 

Due to the advisability of maintaining a response time close to 
the human reaction time (2 seconds), the simulator results show that: a 
limit of 20 should be set on the number of user stations allowed. 

3. 4.2.6 Effects of Varying the Rate of Arrival of Service Requests 

The varying of the rate of arrival of service requests was accomplished 
by varying the average or mean used to generate the times of arrival of 
requests, due to the fact that if the mean is lowered, while the number 
of jobs generated remain the same, the rate of arrival of jobs will increase. 
The mean was varied irom 300 to 50 seconds. An hour was simulated for 
each new mean. The first run was made with a mean arrival time of 300 
seconds and each succeeding run with a mean of 50 seconds less than the 



preceding one . 
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FIGURE 8 (CONTINUED) 
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The output of the simulator analysis, as shown in Figure a, indicates 



that the overload increases as the rate of arrival increases with a corre- 
sponding decrease in computational efficiency 

It ts very apparent that the rate of arrival of service requests 
greatly influences the loading of the system. Fortunately, except for an 
initial surge when the system begins operation for the day, a multi- 
programming system has a fairly low- request arrival rate due to the slow- 
ness of the human thought and action process. 

3.4.3 General Summary of Simulation A nalysis 

The example of the simulator analysis of Algorithm A illustrated 
the flexibility and effectiveness of simulation analysis and outlined 
the method of using the simulator. 

The simulator may be used to analyze and compare several different 
scheduling algorithms in exactly the same manner it was used to analyze 
and compare several variations of Algorithm A. 
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SIMULATOR OUTPUT DATA 
WITH ARRIVAL TIME AVERAGE VARIABLE 
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FIGURE 9 (CONTINUED) 
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4,0 Conclusions and Recommendations 



An investigation of several different methods of analyzing the 
operation of a multiprogramming scheduling algorithm was conducted in 
section 3,1 The advantages and disadvantages of each method were 
determined and a composite investigation procedure was proposed that 
would fully utilize the various favorable characteristics of each method 
and negate the unfavorable ones. This proposed investigation procedure 
as outlined in section 3.2, was demonstrated by the example of the 
investigation of Algorithm A in sections 3.3 and 3.4. 

As shown by the example, the use of the composite investigation 
procedure will bring to light many important facts concerning the opera 
tion of a scheduling algorithm and allow changes to be made to an 
algorithm or a new one adopted without disrupting the system operation. 
The use of the investigation procedure insures that the scheduler invest! 
gated will bring about, a close realization of the system's goals. 

The technique of simulation was show-n to be a very powerful and 
versatile tool for use in the analysis of an algorithm's operation. The 
simulator used for the investigation of the operation of Algorithm A 
demonstrated these qualities by its detailed portrayal of a wide variety 
of system parameters of interest during the simulated operation ot the 
system. One additional feature of the simulator, that was noticed 
during the course of the Investigation, is the detailed insight obtained 
of not only the operation of the scheduler, but also the complete 
multiprogramming system. This insight was gained by the process of 
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formulating and coding the simulator. This experience and knowledge 
of the system would be invaluable to the system's programme: and a 
simulation study o: a system will pay off in this respect alone 

Some of the major problems involved in combining the on-line 
control type user with the normal users of a multiprogramming system, 
were outlined during the heuristic analysis of the algorithm proposed :.y 
the author (Algorithm C) in section 3.3.3. The foremost obstacle being 
the inability of the control type user to allow his service requirements 
to be varied, in order to be compatible with the rest of the users,. It 
must be accepted that all other users bow to the requirements of the 
control user i.e. , the control user's requirements in effect determine 
the scheduler design. With this in mind, Algorithm C and the modifica- 
tions to it, as proposed in section 3 3.3 4, offer reasonable solutions 
to the problem of integrating the control type user into the multiprogram- 
ming system. 

The state of the art in the multiprogramming area may be briefly 
described as just coming out of the hardware limited phase. Heretofore 
it has been extremely difficult to obtain the necessary capabilities in a 
computer system to allow effective multiprogramming. As a result, many 
features that should have been of a hardware nature have, in some cases, 
been implemented by programming (with a resulting increase in overhead) 
cr have been omitted A few of such features are: 

(1) Priority assignment of interrupts and users. 

(2) Quantum determination and interruption 



(3) Memory protection 

With the new large computers being designed for multiprogramming 
applications, this state of affairs will cease to exist and the simple 
scheduling algorithms of today will have to be replaced by more complex 
and versatile ones . The algorithms in present use are entirely satisfactory 
for the present: requirements of 20 to 50 user stations utilizing Teletypes 
and/or displays as i/0 devices and operating in such a manner as to require 
only a small portion of their time in the system to be compute time 

The present scheduling philosophy is capable of providing algorithms 
for use within systems, that concern themselves primarily with satisfying 
a fairly large number of users, whose needs are short bursts of compute 
time, followed by longer intervals of idle or I/O time, A change in 
philosophy, however, appears to be necessary in order to provide adequate 
schedulinq algorithms for use within the military command and control 
type oi multiprogramming system, where the numerous tasks and service 
requirements are both extremely complex and exacting. This change also 
appears necessary when considering the type of system that contains 
multiple facilities (such as multiple processing units, etc.) and requires 
the scheduler to schedule the use of each and every one of the multiple 
units. These problems are too complex for solution by present scheduler 
philosophy and represent, areas of multiprogramming that are fertile fields 
for future research . 

Further research into the use of simulation techniques for analysis 
of scheduling algorithms is needed as the capabilities of simulation are 
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enormous, if utilized properly and integrated with other techniques in 



the correct manner. 

A final recommendation is that the applicability, of dynamic 
programming to the scheduler problem, be investigated thoroughly, as 
the analysis of a scheduler is a form of cost-effectiveness study and 
dynamic programming is very effective in this area. 
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APPENDIX I 



SIM “A MULTIPROGRAMMING SYSTEM SIMULATOR 

Simulation has become a valuable analytic method that can be 
applied in diverse situations. The type of simulation of interest is the 
Monte Carlo Method which is defined in the McGraw Hill Encyclopedia 
of Science and Technology , as "A technique for estimating the solution, 
x, of a numerical mathematical problem by means of an artificial sampling 
experiment ..." The method aptly fits the multiprogramming system 
problem and can produce worthwhile results . The required probability 
distributions associated with users can be determined by general data 
gathering and observation. The use of various algorithms in the Executive 
routine and several hardware configurations in a simulated system subjected 
to a typical loading will produce the data to obtain a measure of effective- 
ness for the various hardware and software configurations. The time and 
expense required to actually evaluate each of the combinations in an 
operating system are prohibitive and the use of simulation techniques 
provides the only realistic approach. 

Program SIM was developed as a general multiprogramming system 
simulator with the emphasis on the time-sharing type of environment. 

Due to the specific nature of the authors’ theses, primary attention was 
given to the Scheduler and Exchanger. The normal performance of other 
specific areas of the Executive routine is assumed and these portions 
treated in a block method. A prime example of this is the Dispatcher 
While it is a critical area of a multiprogramming system no specific 
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characteristics were assumed. When a user has an Input or Output the 



program assumes a waiting status until, due to the incrementing of the 
simulator clock, the action is deemed complete,, The availability of the 
required I/O equipment at all times is assumed. A time-sharing system 
is characterized by frequent man-machine communication and buffered 
I/O is usually impossible due to the step by step nature of the system. 
However, simultaneous I/O by all users, at least to their reactive 
typewriters, must be permitted. The gross Dispatcher treatment provides 
all this and only avoids the complications of particular operations. If 
this area is studied in the future the Dispatcher portion could easily be 
made more detailed and added to the system simulator. 

The job load on the simulated system is created by a job generation 
subroutine (SET). Each job is characterized by six variables, which 
define any job entering the system. Arrival time, the first parameter, 
is assumed to be exponentially distributed on the basis of queuing theory 
concepts and actual observation at System Development Corporation. A 
variable parameter is the mean arrival time expressed in seconds. The 
value of arrival time was determined by taking the natural logarithm of a 
uniformly distributed random number. The second parameter is Load time 
and represents the time required to transfer the binary program from its 
permanent storage to the temporary storage having access to the central 
memory. The next three parameters, Active time, I/O time and Repeats 
define the actual, program operating characteristics. A program once 
loaded into the system is assumed to have an active period followed by 
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an 1/0 period, during which no service is required from the central 
processor. This cycle is repeated until the job is completed or there 
are no further repeats required. Due to the nature of SIM, differences in 
l/O such as tape transfers, searches or outputs to reactive devices are 
not recognized and the I/O operations are grouped together. The sixth 
parameter. Size, completes the job description. Program size is limited 
to a maximum length of one hundred cells less than the full core available 
for operating programs. The last five parameters are determined using a 
Gaussian Random number generator and the mean values received as input. 
A uniform random number generater is used to generate any of ten possible 
job types. The probability of each type job is also received as an input. 
As soon as the job is completed, that is the number of repeats remaining 
is less than zero, a new job is generated for that station. An example 
of the input to SIM is shown at the end of the program contained in 
Appendix II . 

It is possible to obtain a wide variety of output parameters from 
the simulator as it has access to all of the internal system parameters 
concerning the operation of the system in question. These parameters 
may be gathered on a minute, average, total or maximum basis and 
thereby present a picture of the system's operation, in almost: any degree 
of detail desired. 

For the purpose of comparison, the output parameters of one run, 
using a certain hardware and software configuration, may be saved and 
then presented with the results of another run, with changes to the 
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hardware and/or software configuration . The output of the simulator 
may be in a tabular format or modified to a graphical type format, 
which ever is deemed best for comparison purposes „ 

The program operation is cyclic in nature . First, the initial jobs 
are generated and the program constants read in and/or initialized for 
the run,, The main body of the simulator is then entered and the actual 
run commenced. The clock is checked against arrival times of all 
allowed users (maximum 50) and an equality or greater than condition 
sets the action entries of the status table (STAT(X,Y))„ The Scheduler 
then determines which requests for service shall be honored and the 
order in which they shall be honored, i.e„ , queue formation. The 
Scheduler also determines, from the number requesting service, the 
amount of time each user is allowed per cycle. The cycle begins with 
the formation of the queue and ends with termination of the last user's 
quantum. The Scheduler then passes control to the Exchanger. For 
further specific information concerning the operation of the Scheduler 
section of the simulator see Lt. W. G. Wilder's thesis. 

The Exchanger determines the action required by the next user 
in the queue and then LOADS, QUITS or TRANSFERS the users program. 
The actual transfer algorithm is variable and the methods used are dis- 
cussed in detail in Lt. R. R. Hatch's thesis. Regardless of the exact 
method, the required transfers are determined and the effective transfer 
times (TELOAD and TEDUMP) and exchange overhead calculated and 
added to the clock. In the LOAD and QUIT operations the size of 
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the external store, such as a drum, is considered and no load or storage 



full conditions are possible. 

At the completion of a cycle all users in an I/O status are handled 
by decrementing the remaining time by the elapsed cycle time,. Users 
completing I/O are checked for repeats „ If repeats are necessary the 
program is reset to the active mode and if no repeats are required the 
program is terminated by a QUIT command in the next cycle. 

To avoid long idle periods a scheme is used to advance the clock 
to the next active clock time if there are no users desiring service. The 
smallest I/O time remaining (SMALLA) and the nearest arrival time (SMALLB) 
are determined and the smaller of these two added to the clock and a new 
cycle commenced. 

A maximum clock parameter read in terminates the run and the 
capability for recycling is provided . All new parameters may be read in 
or the original parameters may be modified for successive runs. 

A flow diagram of SIM is contained in Figure 1-1 and a copy of the 
actual program is contained in Appendix II . 

Due to the fact that the thesis topics of Lt. W. G. Wilder and 
Lt. R„ R. Hatch are both in the multiprogramming area and a generalized 
multiprogramming simulator could be used in each case, this simulator 
represents the joint efforts of both Lt„ Wilder and Lt. Hatch. 
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APPENDIX II II- I PROGRAM SIM (CONTINUED) 



93 FORMAT (//44X*8HFI GURE 6//) 
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515 FORMAT ( 9F10. 4// ) 
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APPENDIX II II- 5 PROGRAM SIM (CONTINUED) 
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APP.ENOIX II I I- 6 PROGRAM SIM (CONTINUED) 
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IF(TERM) 1530, 1530, 15H0 
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APPENDIX II - II- 9 PROGRAM SIM 
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APPENDIX II II- 10 PROG 8AM SIM (CONTINUED) 
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APPENDIX II 11-12 .PROGRAM SIM (CONTINUED) 
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APPENDIX 11 11-13 . PROGRAM SIM (CONTINUED) 
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